storage system
Deep Learning for Modeling and Dispatching Hybrid Wind Farm Power Generation
Lawrence, Zach, Yao, Jessica, Qin, Chris
Abstract--Wind farms with integrated energy storage, or hybrid wind farms, are able to store energy and dispatch it to the grid following an operational strategy. For individual wind farms with integrated energy storage capacity, data-driven dispatch strategies using localized grid demand and market conditions as input parameters stand to maximize wind energy value. Synthetic power generation data modeled on atmospheric conditions provide another avenue for improving the robustness of data-driven dispatch strategies. T o these ends, the present work develops two deep learning frameworks: COVE-NN, an LSTM-based dispatch strategy tailored to individual wind farms, which reduced annual COVE by 32.3% over 43 years of simulated operations in a case study at the Pyron site; and a power generation modeling framework that reduced RMSE by 9.5% and improved power curve similarity by 18.9% when validated on the Palouse wind farm. T ogether, these models pave the way for more robust, data-driven dispatch strategies and potential extensions to other renewable energy systems. COV E Cost of valued energy. CRPS Continuous ranked probability score. RMSE Root mean squared error.
- North America > United States > California > Los Angeles County > Los Angeles (0.28)
- North America > United States > Texas (0.05)
- North America > United States > Colorado > Jefferson County > Golden (0.04)
- (2 more...)
- Energy > Renewable > Wind (1.00)
- Government > Regional Government > North America Government > United States Government (0.68)
AVS: A Computational and Hierarchical Storage System for Autonomous Vehicles
Wang, Yuxin, He, Yuankai, Shi, Weisong
Autonomous vehicles (AVs) are evolving into mobile computing platforms, equipped with powerful processors and diverse sensors that generate massive heterogeneous data, for example 14 TB per day. Supporting emerging third-party applications calls for a general-purpose, queryable onboard storage system. Yet today's data loggers and storage stacks in vehicles fail to deliver efficient data storage and retrieval. This paper presents AVS, an Autonomous Vehicle Storage system that co-designs computation with a hierarchical layout: modality-aware reduction and compression, hot-cold tiering with daily archival, and a lightweight metadata layer for indexing. The design is grounded with system-level benchmarks on AV data that cover SSD and HDD filesystems and embedded indexing, and is validated on embedded hardware with real L4 autonomous driving traces. The prototype delivers predictable real-time ingest, fast selective retrieval, and substantial footprint reduction under modest resource budgets. The work also outlines observations and next steps toward more scalable and longer deployments to motivate storage as a first-class component in AV stacks.
- North America > United States (0.28)
- Europe > Slovenia > Drava > Municipality of Benedikt > Benedikt (0.04)
- Transportation > Ground > Road (1.00)
- Information Technology (1.00)
- Automobiles & Trucks (1.00)
- Information Technology > Artificial Intelligence > Robots > Autonomous Vehicles (1.00)
- Information Technology > Architecture (1.00)
StorageXTuner: An LLM Agent-Driven Automatic Tuning Framework for Heterogeneous Storage Systems
Lin, Qi, Zhang, Zhenyu, Thakkar, Viraj, Sun, Zhenjie, Zheng, Mai, Cao, Zhichao
Automatically configuring storage systems is hard: parameter spaces are large and conditions vary across workloads, deployments, and versions. Heuristic and ML tuners are often system specific, require manual glue, and degrade under changes. Recent LLM-based approaches help but usually treat tuning as a single-shot, system-specific task, which limits cross-system reuse, constrains exploration, and weakens validation. We present StorageXTuner, an LLM agent-driven auto-tuning framework for heterogeneous storage engines. StorageXTuner separates concerns across four agents - Executor (sandboxed benchmarking), Extractor (performance digest), Searcher (insight-guided configuration exploration), and Reflector (insight generation and management). The design couples an insight-driven tree search with layered memory that promotes empirically validated insights and employs lightweight checkers to guard against unsafe actions. We implement a prototype and evaluate it on RocksDB, LevelDB, CacheLib, and MySQL InnoDB with YCSB, MixGraph, and TPC-H/C. Relative to out-of-the-box settings and to ELMo-Tune, StorageXTuner reaches up to 575% and 111% higher throughput, reduces p99 latency by as much as 88% and 56%, and converges with fewer trials.
- North America > United States > Colorado > Denver County > Denver (0.04)
- North America > United States > New York > Suffolk County > Stony Brook (0.04)
- North America > United States > Iowa (0.04)
- (5 more...)
Intent-Driven Storage Systems: From Low-Level Tuning to High-Level Understanding
Bergman, Shai, Song, Won Wook, Cavigelli, Lukas, Berestizshevsky, Konstantin, Zhou, Ke, Zhang, Ji
Existing storage systems lack visibility into workload intent, limiting their ability to adapt to the semantics of modern, large-scale data-intensive applications. This disconnect leads to brittle heuristics and fragmented, siloed optimizations. To address these limitations, we propose Intent-Driven Storage Systems (IDSS), a vision for a new paradigm where large language models (LLMs) infer workload and system intent from unstructured signals to guide adaptive and cross-layer parameter reconfiguration. IDSS provides holistic reasoning for competing demands, synthesizing safe and efficient decisions within policy guardrails. We present four design principles for integrating LLMs into storage control loops and propose a corresponding system architecture. Initial results on FileBench workloads show that IDSS can improve IOPS by up to 2.45X by interpreting intent and generating actionable configurations for storage components such as caching and prefetching. These findings suggest that, when constrained by guardrails and embedded within structured workflows, LLMs can function as high-level semantic optimizers, bridging the gap between application goals and low-level system control. IDSS points toward a future in which storage systems are increasingly adaptive, autonomous, and aligned with dynamic workload demands.
- Europe > Switzerland > Zürich > Zürich (0.15)
- North America > United States > New York > New York County > New York City (0.05)
- North America > United States > North Carolina (0.04)
- (9 more...)
- Workflow (0.54)
- Research Report > New Finding (0.34)
Topology-Aware and Highly Generalizable Deep Reinforcement Learning for Efficient Retrieval in Multi-Deep Storage Systems
Li, Funing, Tian, Yuan, Noortwyck, Ruben, Zhou, Jifeng, Kuang, Liming, Schulz, Robert
In modern industrial and logistics environments, the rapid expansion of fast delivery services has heightened the demand for storage systems that combine high efficiency with increased density. Multi-deep autonomous vehicle storage and retrieval systems (AVS/RS) present a viable solution for achieving greater storage density. However, these systems encounter significant challenges during retrieval operations due to lane blockages. A conventional approach to mitigate this issue involves storing items with homogeneous characteristics in a single lane, but this strategy restricts the flexibility and adaptability of multi-deep storage systems. In this study, we propose a deep reinforcement learning-based framework to address the retrieval problem in multi-deep storage systems with heterogeneous item configurations. Each item is associated with a specific due date, and the objective is to minimize total tardiness. To effectively capture the system's topology, we introduce a graph-based state representation that integrates both item attributes and the local topological structure of the multi-deep warehouse. To process this representation, we design a novel neural network architecture that combines a Graph Neural Network (GNN) with a Transformer model. The GNN encodes topological and item-specific information into embeddings for all directly accessible items, while the Transformer maps these embeddings into global priority assignments. The Transformer's strong generalization capability further allows our approach to be applied to storage systems with diverse layouts. Extensive numerical experiments, including comparisons with heuristic methods, demonstrate the superiority of the proposed neural network architecture and the effectiveness of the trained agent in optimizing retrieval tardiness.
- Europe > Germany > Baden-Württemberg > Stuttgart Region > Stuttgart (0.04)
- Africa > Eswatini > Manzini > Manzini (0.04)
- Europe > Germany > Bavaria > Upper Bavaria > Munich (0.04)
- Asia > China > Hubei Province (0.04)
- Transportation (0.67)
- Leisure & Entertainment > Games > Computer Games (0.46)
NEURODNAAI: Neural pipeline approaches for the advancing dna-based information storage as a sustainable digital medium using deep learning framework
Thakur, Rakesh, Singh, Lavanya, Yashika, null, Bundawala, Manomay, Kumar, Aruna
DNA is a promising medium for digital information storage for its exceptional density and durability. While prior studies advanced coding theory, workflow design, and simulation tools, challenges such as synthesis costs, sequencing errors, and biological constraints (GC-content imbalance, homopolymers) limit practical deployment. To address this, our framework draws from quantum parallelism concepts to enhance encoding diversity and resilience, integrating biologically informed constraints with deep learning to enhance error mitigation in DNA storage. Our results show that traditional prompting or rule-based schemes fail to adapt effectively to realistic noise, whereas NeuroDNAAI achieves superior accuracy. Experiments on benchmark datasets demonstrate low bit error rates for both text and images. By unifying theory, workflow, and simulation into one pipeline, NeuroDNAAI enables scalable, biologically valid archival DNA storage. The rapid increase in global data generation has placed unprecedented pressure on traditional storage media, including magnetic tapes, hard disks, and solid-state drives. These technologies are constrained in terms of density, durability, and sustainability, often degrading within decades and necessitating frequent migration. At the same time, forecasts indicate that the volume of digital data will soon surpass the capacity of existing storage infrastructure, creating an urgent demand for alternative paradigms. DNA has emerged as a promising medium for information storage due to its extremely high density, long-term stability, and universal biological accessibility. Despite this extraordinary theoretical potential, practical adoption remains hindered by challenges in synthesis, sequencing, and error correction. Errors such as substitutions, insertions, and deletions complicate reliable retrieval, thereby motivating the development of novel methods capable of tolerating or correcting these distortions. In response to these challenges, the present work proposes a modular end-to-end framework that simulates the DNA storage pipeline and introduces a Transformer-based neural decoder for robust data reconstruction. Within this system, digital information (in this case, MNIST images) is encoded into DNA sequences, passed through a configurable noise model that simulates synthesis and sequencing errors, and subsequently reconstructed using an encoder-decoder architecture.
- Research Report > New Finding (0.54)
- Research Report > Promising Solution (0.34)
Smash: Flexible, Fast, and Resource-Efficient Placement and Lookup of Distributed Storage
Large-scale distributed storage systems, such as object stores, usually apply hashing-based placement and lookup methods to achieve scalability and resource efficiency. However, when object locations are determined by hash values, placement becomes inflexible, failing to optimize or satisfy application requirements such as load balance, failure tolerance, parallelism, and network/system performance. This work presents a novel solution to achieve the best of two worlds: flexibility while maintaining cost-effectiveness and scalability. The proposed method Smash is an object placement and lookup method that achieves full placement flexibility, balanced load, low resource cost, and short latency. Smash uses a recent space-efficient data structure and applies it to object-location lookups. We implement Smash as a prototype system and evaluate it in a public cloud. The analysis and experimental results show that Smash achieves full placement flexibility, fast storage operations, fast recovery from node dynamics, and lower DRAM cost (less than 60%) compared to existing hash-based solutions, such as Ceph and MapX.
- Research Report > New Finding (0.34)
- Research Report > Promising Solution (0.34)
Technical Perspective: Where Is My Data?
Membership in ACM includes a subscription to Communications of the ACM (CACM), the computing industry's most trusted source for staying connected to the world of advanced computing. Technical Perspective: Where Is My Data? Smash: Flexible, Fast, and Resource-Efficient Placement and Lookup of Distributed Storage, by Yi Liu et al., introduces a storage system that is the first of its kind to apply minimal perfect hashing (MPH) to storage research. When making a storage system distributed across many machines, designers are faced with a critical question: Where is my data? At the heart of every storage system is a mechanism for determining where to find data based on an identifier.
Harmonia: A Multi-Agent Reinforcement Learning Approach to Data Placement and Migration in Hybrid Storage Systems
Nadig, Rakesh, Arulchelvan, Vamanan, Bera, Rahul, Shahroodi, Taha, Singh, Gagandeep, Kakolyris, Andreas, Sadrosadati, Mohammad, Park, Jisung, Mutlu, Onur
Hybrid storage systems (HSS) integrate multiple storage devices with diverse characteristics to deliver high performance and capacity at low cost. The performance of an HSS highly depends on the effectiveness of two key policies: (1) the data-placement policy, which determines the best-fit storage device for incoming data, and (2) the data-migration policy, which dynamically rearranges stored data (i.e., prefetches hot data and evicts cold data) across the devices to sustain high HSS performance. Prior works optimize either data placement or data migration in isolation, which leads to suboptimal HSS performance. Unfortunately, no prior work tries to optimize both policies together. Our goal is to design a holistic data-management technique that optimizes both data-placement and data-migration policies to fully exploit the potential of an HSS, and thus significantly improve system performance. We propose Harmonia, a multi-agent reinforcement learning (RL)-based data-management technique that employs two lightweight autonomous RL agents, a data-placement agent and a data-migration agent, that adapt their policies for the current workload and HSS configuration while coordinating with each other to improve overall HSS performance. We evaluate Harmonia on real HSS configurations with up to four heterogeneous storage devices and seventeen data-intensive workloads. On performance-optimized (cost-optimized) HSS with two storage devices, Harmonia outperforms the best-performing prior approach by 49.5% (31.7%) on average. On an HSS with three (four) devices, Harmonia outperforms the best-performing prior work by 37.0% (42.0%) on average. Harmonia's performance benefits come with low latency (240ns for inference) and storage overheads (206 KiB in DRAM for both RL agents combined). We will open-source Harmonia's implementation to aid future research on HSS.
- Europe > Switzerland > Zürich > Zürich (0.04)
- Asia > Middle East > Republic of Türkiye > Batman Province > Batman (0.04)
- Information Technology (0.67)
- Energy (0.46)
A Data-Driven Machine Learning Approach for Predicting Axial Load Capacity in Steel Storage Rack Columns
Mammadli, Bakhtiyar, Yazici, Casim, Gürbüz, Muhammed, Kocaman, İrfan, Dominguez-Gutierrez, F. Javier, Özkal, Fatih Mehmet
In this study, we present a machine learning (ML) framework to predict the axial load-bearing capacity, (kN), of cold-formed steel structural members. The methodology emphasizes robust model selection and interpretability, addressing the limitations of traditional analytical approaches in capturing the nonlinearities and geometrical complexities inherent to buckling behavior. The dataset, comprising key geometric and mechanical parameters of steel columns, was curated with appropriate pre-processing steps including removal of non-informative identifiers and imputation of missing values. A comprehensive suite of regression algorithms, ranging from linear models to kernel-based regressors and ensemble tree methods was evaluated. Among these, Gradient Boosting Regression exhibited superior predictive performance across multiple metrics, including the coefficient of determination (R2), root mean squared error (RMSE), and mean absolute error (MAE), and was consequently selected as the final model. Model interpretability was addressed using SHapley Additive exPlanations (SHAP), enabling insight into the relative importance and interaction of input features influencing the predicted axial capacity. To facilitate practical deployment, the model was integrated into an interactive, Python-based web interface via Streamlit. This tool allows end-users-such as structural engineers and designers, to input design parameters manually or through CSV upload, and to obtain real-time predictions of axial load capacity without the need for programming expertise. Applied to the context of steel storage rack columns, the framework demonstrates how data-driven tools can enhance design safety, streamline validation workflows, and inform decision-making in structural applications where buckling is a critical failure mode
- Asia > Middle East > Republic of Türkiye > Erzurum Province > Erzurum (0.04)
- South America > Colombia > Huila Department > Neiva (0.04)
- Europe > Poland (0.04)
- Materials (1.00)
- Energy (1.00)
- Information Technology > Security & Privacy (0.93)